Three dimensional measurement apparatus

ABSTRACT

A three-dimensional measurement apparatus includes an optical system for scanning a reference beam across a target object to be measured, a light sensor which receives light reflected from the target object, and a processor for calculating a three-dimensional shape of the target object from the received light. An image for calculating the three-dimensional shape of the target object and an image for displaying the target object are both captured by the same sensor. The displayed image is a grayscale image that is based on a centroid that is calculated from multiple data samples taken for each pixel in the image as the target is being scanned.

This disclosure is based upon, and claims priority from, provisionalU.S. patent application No. 60/100,884, filed Sep. 23, 1998, andJapanese Application No. 10-079431, filed Mar. 26, 1998, the contents ofwhich are incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates to a three-dimensional measurementapparatus, and more particularly to a so-called active typethree-dimensional measurement apparatus that captures three-dimensionaldata by projecting a reference beam on a target object and receiving itsreflected light.

BACKGROUND OF THE INVENTION

In the prior art, so-called active type three-dimensional measurementapparatuses have been known that capture three-dimensional data byprojecting a reference beam on a target object and receiving itsreflected light. Among these types of devices, an apparatus that cangenerate both a range image (the image used to measure distance and tocalculate the three-dimensional shape of the target object) and a colorimage (the image used to display the target object) is known, asdisclosed for example, in Japanese Patent Unexamined Publication No.9-145319.

FIG. 19 is a block diagram showing the configuration of such an activetype three-dimensional measurement apparatus. Referring to the figure, abeam splitter (beam-splitting prism) 52 is constructed with a colorseparation film (dichroic mirror) 521, two prisms 522 and 523sandwiching the color separation film 521, a range image capturing CCDsensor 53 provided on the emergent face of the prism 522, and a colorimage capturing CCD sensor 54 provided on the emergent surface of theprism 523.

A target object is scanned by a reference beam emitted from asemiconductor laser, and the light reflected by the target object entersa light receiving lens 51 a. The light that enters the light receivinglens 5 la passes through the prism 522 and reaches the color separationfilm 521. Light U0 lying in the oscillation wavelength region of thesemiconductor laser is reflected by the color separation film 521 anddirected toward the range image capturing CCD sensor 53. On the otherhand, light C0 transmitted through the color separation film 521 passesthrough the prism 523 and enters the color image capturing CCD sensor54.

The range image capturing CCD sensor 53 is driven by a range imagecapturing CCD driver 204. The color image capturing CCD sensor 54 isdriven by a color image capturing CCD driver 203. The output of therange image capturing CCD sensor 53 is processed by an A/D converter (anoutput processing circuit) 202, and then stored in a range image framememory 206. The output of the color image capturing CCD sensor 54 isprocessed by an A/D converter (an output processing circuit) 201, andthen stored in a color image frame memory 205.

The prior art active type three-dimensional measurement apparatusdescribed above has exhibited the following limitations, due to the useof the two CCD sensors 53 and 54.

(1) The mounting positions of the two CCD sensors must be adjusted veryprecisely so as to eliminate misregistration between the range image andthe color image.

(2) Since near infrared light is primarily used for the reference beam,it is necessary to produce a prism that can separate incident light intonear infrared light for the range image and visible light for the colorimage.

(3) The quality of the color image is dependent on the spectralcharacteristics of the prism that is used.

The present invention has been devised to address the above-listedlimitations, and one object of the invention is to solve the problemsassociated with the use of two sensors in a three-dimensionalmeasurement apparatus.

SUMMARY OF THE INVENTION

To achieve the above object, according to one aspect of the presentinvention, a three-dimensional measurement apparatus comprisesprojecting means for projecting a reference beam toward a target objectto be measured, light receiving means for receiving light reflected fromthe target object, and calculating means for calculating athree-dimensional shape of the target object from the received reflectedlight. In this apparatus, an image for calculating the three-dimensionalshape of the target object and an image for displaying the target objectare captured by the same sensor.

According to the present invention, since the image for calculating thethree-dimensional shape of the target object and the image fordisplaying the target object are both captured by the same sensor, thereis no need to adjust the mounting positions of two sensors, which wasthe case with the prior art. Furthermore, there is no need to produce aprism capable of separating incident light into near infrared light andvisible light, and therefore, the quality of the color image can beimproved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing the configuration of a measurement systemaccording to a first embodiment of the present invention.

FIGS. 2a and 2 b are diagrams showing an external view of athree-dimensional camera.

FIG. 3 is a block diagram showing the functional configuration of thethree-dimensional camera.

FIG. 4 is a diagram showing the configuration of a filter, a filterswitching mechanism, and their peripheral circuitry according to thefirst embodiment.

FIG. 5 is a block diagram of a three-dimensional camera according to asecond embodiment of the present invention.

FIG. 6 is a diagram showing the configuration of a filter, a filterswitching mechanism, and their peripheral circuitry according to thesecond embodiment.

FIG. 7 is a block diagram showing the configuration of athree-dimensional camera according to a third embodiment of the presentinvention.

FIG. 8 is a diagram showing the configuration of a filter and itsperipheral circuitry according to the third embodiment.

FIGS. 9a and 9 b are schematic diagrams showing the construction of aprojection lens system.

FIGS. 10a and 10 b are diagrams illustrating the principle ofthree-dimensional position calculations in the measurement system.

FIG. 11 is a diagram showing a sensor readout range.

FIGS. 12a and 12 b are diagrams showing the relationship between linesand frames on a sensor imaging surface.

FIG. 13 is a diagram showing how incident light data for each frame isstored in a memory.

FIG. 14 is a diagram showing how incident light data for an additionalframe is stored in the memory.

FIG. 15 is a diagram showing how incident light data for furtheradditional frames is stored in the memory.

FIG. 16 is a block diagram showing the configuration of a centroidcalculation circuit.

FIG. 17 is a diagram showing the concept of data transfer timing.

FIG. 18 is a diagram showing the concept of a temporal centroid.

FIG. 19 is a diagram for explaining the configuration of an active typethree-dimensional measurement apparatus according to the prior art.

DETAILED DESCRIPTION

FIG. 1 is a diagram showing the configuration of a measurement system 1according to a first embodiment of the present invention. Referring tothe figure, the measurement system 1 comprises a three-dimensionalcamera (range finder) 2 which makes stereoscopic measurements using aslit ray projection method, and a host system 3 which processes outputdata from the three-dimensional camera 2.

The three-dimensional camera 2 outputs a two-dimensional imagedescribing color information of an object Q and data necessary forcalibration, together with measurement data (slit image data)identifying the three-dimensional positions of a plurality of samplingpoints on the object Q. The host 3 performs calculations to obtain thecoordinates of the sampling points using a triangulation method.

The host 3 is a computer system comprising a CPU 3 a, a display 3 b, akeyboard 3 c, a mouse 3 d, etc. The CPU 3 a incorporates software forprocessing the measurement data. Both on-line data transfer and off-linedata transfer, using a removable recording medium 4, are possible as amethod of data transfer between the host 3 and the three-dimensionalcamera 2. Examples of the recording medium 4 include magneto-opticaldisks (MOs), mini-disks (MDs), and memory cards.

FIGS. 2a and 2 b are diagrams showing an external view of thethree-dimensional camera 2. A projection window 20 a and a lightreceiving window 20 b are provided in the front panel of a housing 20.The projection window 20 a is located upward of the light receivingwindow 20 b. A slit ray U (a strip of laser beam with a prescribed widthof w) emitted from an internal optical unit OU passes through theprojection window 20 a and is directed toward an object to be measured(the subject or the target object). The radiating angle φ along thelengthwise direction M1 of the slit ray U is fixed. Part of the slit rayU reflected from the surface of the object passes through the lightreceiving window 20 b and enters the optical unit OU. The optical unitOU is equipped with a two-axis adjusting mechanism for optimizing therelative relationship between the projection axis and the lightreceiving axis.

On the top panel of the housing 20 are provided zooming buttons 25 a and25 b, manual focusing buttons 26 a and 26 b, and a shutter button 27. Asshown in FIG. 2(b), a liquid crystal display 21, cursor buttons 22, aselect button 23, a cancel button 24, an analog output terminal 32, adigital output terminal 33, and an insertion slot 30 a for the recordingmedium 4 are provided on the rear panel of the housing 20.

The liquid crystal display (LCD) 21 is used as an electronic viewfinderas well as an operation screen display means. The camera operator canset up the shooting mode by using the buttons 22 to 24 on the rearpanel. Measurement data is output from the digital output terminal 33,and a two-dimensional image signal is output, for example in the NTSCformat, from the analog output terminal 32. The digital output terminal33 is, for example, a SCSI terminal.

FIG. 3 is a block diagram showing the functional configuration of thethree-dimensional camera 2. In the figure, solid-line arrows indicateelectrical signal flows, and dotted-line arrows show light paths.

The three-dimensional camera includes two optical systems 40 and 50, onefor projection and the other for light reception, which togetherconstitute the optical unit OU. In the optical system 40, a laser beamwith a wavelength of 670 nm emitted from a semiconductor laser (LD) 41passes through a projection lens system 42 to form the slit ray U whichis deflected by a galvanometer mirror (scanning means) 43. A driver 44for the semiconductor laser 41, a driving system 45 for the projectionlens system 42, and a driving system 46 for the galvanometer mirror 43are controlled by a system controller 61.

In the optical system 50, incident light converged by a zoom unit 51enters a filter 80A. The detail of the filter 80A will be describedlater. The incident light passes through the filter 80A and enters acolor measuring sensor (CCD) 53 a. The zoom unit 51 is of the internalfocusing type, and a portion of the incident light is used for autofocusing (AF). The AF function is implemented using an AF sensor 57, alens controller 58, and a focusing driving system 59. A zooming drivingsystem 60 is provided for motor-driven zooming.

Imaging information captured by the color measuring sensor 53 a istransferred to a memory 63 or a color processing circuit 67 insynchronism with a clock signal from a driver 55. The imaginginformation subjected to color processing in the color processingcircuit 67 is quantized in a digital image generator 68 and stored in acolor image memory 69. After that, the color image data is transferredfrom the color image memory 69 to a SCSI controller 66, and is outputonline via the digital output terminal 33 or stored on the recordingmedium 4 in association with the measurement data. The imaginginformation stored in the memory 69 is also output online via an NTSCconversion circuit 70 and the analog output terminal 32. In an alternateembodiment (not shown), analog imaging information produced by the colorprocessing circuit can be directly supplied to the NTSC conversioncircuit 70, bypassing the digital image generator 69.

The color image is an image with the same angle of view as the rangeimage captured by the color measuring sensor 53 a, and is used asreference information during application processing at the host 3.Processing that utilizes the color information includes, for example,processing for generating a three-dimensional geometric model bycombining multiple sets of measurement data having different camerafocuses, processing for decimating unnecessary vertexes of thethree-dimensional geometric model, etc. The system controller 61 issuesinstructions to a character generator (not shown), to display propercharacters and symbols on the screen of the LCD 21.

An output processing circuit 62 includes an amplifier for amplifying anoptical-to-electrical converted signal representing each pixel g whichis output from the color measuring sensor 53 a, and an A/D converter forconverting the optical-to-electrical signal to 8-bit incident lightdata. The memory 63 is a read-write memory, and stores the incidentlight data output from the output processing circuit 62. In theillustrated embodiment, if each pixel of an image is represented by onebyte of data, the memory 63 has a storage capacity of 200×32×33 bytes,for reasons which will become apparent. A memory control circuit 63Aspecifies addresses for reading and writing the memory 63.

A centroid calculation circuit 73, based on the incident light datastored in the memory 63, generates a grayscale image corresponding tothe shape of the target object and supplies the image to a displaymemory 74, and also calculates data for calculating three-dimensionalpositions and supplies the data to an output memory 64. The grayscaleimage stored in the display memory 74, and the color image stored in thecolor image memory 69, are displayed on the screen of the LCD 21.

The NTSC conversion circuit 70 includes a video D/A (analog imagegenerating circuit). The filter 80A is switched by a filter switchingmechanism 81A.

FIG. 4 is a diagram showing the details of the filter 80A, the filterswitching mechanism 81A, and their peripheral circuitry. Referring tothe figure, the filter 80A contains an IR (infrared) cutoff filter 80 aand a band pass filter 80 b. The filter switching mechanism 81A switchesthe filter 80A so that the light introduced through the light receivinglens 51 a in the zoom unit 51 is input to the color measuring sensor 53a either through the IR cutoff filter 80 a or through the band passfilter 80 b. More specifically, when capturing the range image, the bandpass filter 80 b, whose pass band corresponds to the wavelength of thereference beam, is positioned in the optical path between the lens 51 aand the sensor 53 a, and when capturing the color image, the IR cutofffilter 80 a is used. The color measuring sensor 53 a is driven by thedriver 55. The range image is latched into the memory 63 after beingprocessed by the output processing circuit, as previously described. Thecolor image data is supplied to the color processing circuit 67.

In this way, in the measurement system of the present embodiment, therange image and the color image are displaced in time so that bothimages can be captured using the same measuring sensor 53 a. When thereis little or no motion in the target object, this displacement in timedoes not present a particular problem in performing the measurements.

Depending on the light receiving lens 51 a that is used and thewavelength of the reference beam, there may arise a displacement betweenthe focal point of the visible light and the focal point of thereference beam. If this happens, adjustments can be made by varying thethicknesses of the filters 80 a and 80 b so that the focal pointscoincide with each other.

A sensor having at least one channel which has a light sensitivity inthe wavelength region of the reference beam is used as the colormeasuring sensor (CCD) 53 a. When a particular channel of the colormeasuring sensor 53 a has a light sensitivity in the wavelength regionof the reference beam, only that particular channel is used forcapturing the range image. For example, only the R (red) channel mightbe used for capturing the range image. When all the channels of thecolor measuring sensor 53 a have a light sensitivity in the wavelengthregion of the reference beam (provided that the degree of sensitivitydoes not affect the measurements), all the channels (R, G (green), and B(blue)) are used for capturing the range image.

The filter 80A can be placed anywhere upstream of the color measuringsensor 53 a. It is, however, desirable that the filter 80A be placedbetween the light receiving lens 51 a and the color measuring sensor 53a. By so doing, the areas of the filters 80 a and 80 b can be reduced,thereby decreasing the burden of the filter switching mechanism 81A.

The present embodiment has been described with a color CCD of an (R, G,B) configuration as an example, but it will be appreciated that a colorCCD of a (G, Cy, Ye, Mg) configuration may be used as the colormeasuring sensor 53 a.

When the arrangement of the present embodiment is employed, there is noneed to adjust the mounting positions of two sensors, which was the casewith the prior art, since the range image and the color image are bothread using the same sensor. Furthermore, there is no need to produce aprism capable of separating the incident light into near infrared lightand visible light, and thus the quality of the color image can beimproved. Moreover, since only one sensor is used, it becomes possibleto reduce the peripheral circuitry compared with the prior art.

FIG. 5 is a diagram showing the configuration of a three-dimensionalcamera according to a second embodiment of the present invention. Thethree-dimensional camera of the second embodiment differs from the firstembodiment in the following points. In the second embodiment, the filter80A (see FIG. 3) of the first embodiment is replaced by a filter 80B.Further, the filter switching mechanism 81A is replaced by a filterswitching mechanism 81B. Also, the color measuring sensor 53 a isreplaced by a monochrome measuring sensor (monochrome CCD) 53 b. In thesecond embodiment, the output of the monochrome measuring sensor 53 b isinput directly to the digital image generator 68. That is, the colorprocessing circuit 67 is not provided in the second embodiment.

FIG. 6 is a diagram showing the configuration of the filter 80B, themonochrome measuring sensor 53 b, and their peripheral circuitry. Thefilter 80B contains a blue filter 80 c, a green filter 80 d, a redfilter 80 e, and a band pass filter 80 f. The blue filter 80 c, thegreen filter 80 d, and the red filter 80 e are filters for the colorimage. The filter switching mechanism 81B performs control by switchingbetween the respective filters so that the reflected light introducedthrough the light receiving lens 51 a impinges upon the monochromemeasuring sensor 53 b after passing through the appropriate filter. Themonochrome measuring sensor 53 b is driven by the driver 55, and theoutput of the monochrome measuring sensor 53 b is supplied to the memory63 via the output processing circuit 62 or to the digital imagegenerator 68.

In the present embodiment, the filter switching mechanism 81B performsfilter switching so that when capturing the range image, for example,the band pass filter, whose pass band corresponds to the wavelength ofthe reference beam, is used. On the other hand, when capturing the colorimage, the R, G, and B filters 80 c to 80 e are used in sequence tocapture three images corresponding to the respective colors.

In the present embodiment, since a monochrome measuring sensor 53 b canbe used, the number of driver circuits and processing circuitsassociated with the sensor can be reduced in comparison with the firstembodiment, thus serving to reduce the apparatus cost.

The above embodiment has been described using R, G, B color separationfilters, but instead, C, M, Y color separation filters may be used.Further, if a col or image is to be reproduced, other colorconfigurations may be used.

FIG. 7 is a diagram showing the configuration of a three-dimensionalcamera according to a third embodiment of the present invention. Thethree-dimensional camera of the present embodiment differs from thefirst embodiment in the following points. A color image is not displayedon the three-dimensional camera of the present embodiment. Only amonochrome luminance image is displayed on the LCD 21. In the presentembodiment, a filter 80C and a monochrome measuring sensor 53 c (or acolor measuring sensor) are used in place of the filter 80A and colormeasuring sensor 53 a shown in FIG. 3. The filter switching mechanism81A, color processing circuit 67, digital image generator 68, colorimage memory 69, NTSC converter 70, and analog output terminal 32 shownin FIG. 3 are not used in the present embodiment.

FIG. 8 is a diagram illustrating the configuration of the filter 80C,the monochrome measuring sensor 53 c, and their peripheral circuitry.The filter 80C consists only of a band pass filter 80 g. The reflectedlight from the target object, introduced through the light receivinglens 51 a, enters the monochrome measuring sensor 53 c via the band passfilter 80 g. The monochrome measuring sensor 53 c is driven by thedriver 55. The output of the monochrome measuring sensor 53 c issupplied to the memory 63 via the output processing circuit 62.

The monochrome measuring sensor 53 c, a CCD area sensor, has anintegrating region and an accumulating region; when integratingoperations in the integrating region are completed, charges aretransferred at once to the accumulating region from which the chargesare sequentially output to external circuitry. In the presentembodiment, image data captured by the monochrome measuring sensor 53 cis used to produce the range image and the display image. That is, inthe present embodiment also, the range image and the display image areinput using a single sensor.

Data used for the calculation of the centroid, i.e., Sxi, is used as themonochrome luminance image for display. A method of obtaining Sxi andthe reason that it can be used as the monochrome luminance image fordisplay will be described.

FIGS. 9a and 9 b are schematic diagrams showing the construction of theprojection lens system 42. FIG. 9a is a front view, and FIG. 9b is aside view. The projection lens system 42 consists of three lenses, thatis, a collimator lens 421, a variator lens 422, and an expander lens423. Optical processing is performed in the following sequence on thelaser beam emitted from the semiconductor laser 41 to obtain a suitableslit ray U. First, the beam is collimated by the collimator lens 421.Next, the beam diameter of the laser beam is adjusted by the variatorlens 422. Finally, the beam is expanded along the slit length directionM1 by the expander lens 423.

The variator lens 422 is provided so that the slit ray U of widthcorresponding to three or more pixels is projected on the measuringsensor 53 c regardless of the object distance and the angle of view.Under direction of the system controller 61, the driving system 45 movesthe variator lens 422 in such a manner as to maintain the width, w, ofthe slit ray U constant on the measuring sensor 53 c. The zoo m unit 51at the light receiving side moves in interlocking fashion with thevariator lens 422.

When the slit length is expanded prior to deflection by the galvanometermirror 43, distortion of the slit ray U can be reduced more effectivelythan when the expansion is done after the deflection. Further, thegalvanometer mirror 43 can be reduced in size by disposing the expanderlens 423 in the final stage of the projection lens system 42, that is,at a position closer to the galvanometer mirror 43.

FIGS. 10a and 10 b are diagrams illustrating the principle ofthree-dimensional position calculations in the measurement system 1. Inthe diagrams, only five samplings of the amount of incident light areshown to facilitate an understanding.

A slit ray U which is wide enough to cover a plurality of pixels on animaging surface S2 of the measuring sensor 53 c, is directed to theobject Q. More specifically, the width of the slit ray U is set equal tothat of five pixels. The slit ra y U is deflected from the top towardthe bottom in FIG. 10 so that it moves on the imaging surface S2 by onepixel pitch pv in each sampling cycle, and the object Q is thus scanned.At the end of each sampling cycle, incident light data(optical-to-electrical converted information) for one frame is outputfrom the monochrome measuring sensor 53 c. In practice, the deflectionis performed at constant angular velocity.

When attention is paid to one particular pixel g on the imaging surfaceS2, in the present embodiment the incident light data is obtained 32times by the 32 samplings taken during the scanning. The timing(temporal centroid Npeak or centroid ip) at which the optical axis ofthe slit ray U passes the object surface region ag opposing the givenpixel g, is obtained by calculating the centroid of the incident lightdata obtained 32 times.

When the surface of the object Q is flat, and there is no noiseattributable to the characteristics of the optical system, the amount ofincident light on the given pixel g increases at the time when the slitray passes, as shown in FIG. 10b, and its distribution usually resemblesa Gaussian distribution. When the amount of incident light is maximum ata time intermediate the n-th sampling and the immediately preceding(n−1)th sampling, as shown in the figure, that time substantiallycoincides with the temporal centroid Npeak.

The position (coordinates) of the object Q is calculated by means ofwell-known triangulation principles, based on the relationship betweenthe intensity of the slit ray radiation and the direction of theincident slit ray on the given pixel at the above-obtained temporalcentroid. This achieves a measurement with a resolution higher than theresolution defined by the pixel pitch pv on the imaging surface.

The amount of incident light on the given pixel g depends on thereflectance of the object Q. However, the relative ratio between eachsampled incident light amount is constant regardless of the absoluteamount of incident light. That is, the lightness or darkness of theobject color does not affect the accuracy of the centroid peakdetermination, and hence the depth measurement.

FIG. 11 is a diagram showing the measurement region of the monochromemeasuring sensor 53 c. As shown in FIG. 11, the readout of one framefrom the monochrome measuring sensor 53 c is done, not using the entireimaging surface S2, but using only the effective light receiving region(a zonal image) Ae, which comprises a portion of the imaging surface S2,to achieve high speed reading. The effective light receiving region Aeis the region of the imaging surface S2 that corresponds to themeasurable distance range of the object Q at a particular radiationtiming of the slit ray U, and shifts by one pixel for each frame as theslit ray U is deflected. In the present embodiment, the number of pixelsin the shift direction of the effective light receiving region Ae isfixed at 32. The method of reading only a portion of the image capturedby a CCD area sensor is disclosed in Japanese Patent UnexaminedPublication No. 7-174536, the disclosure of which is incorporated hereinby reference.

FIGS. 12a and 12 b are diagrams showing the relationship between a lineand a frame on the imaging surface S2 of the monochrome measuring sensor53 c, and FIGS. 13 to 15 are diagrams showing how the incident lightdata of the respective frames are stored in the memory 63.

As shown in FIGS. 12a and 12 b, frame 1, which is the first frame on theimaging surface S2, contains incident light data for 32×200 pixels (fromline 1 to line 32). Frame 2 is from line 2 to line 33, and frame 3 isfrom line 3 to line 34, the region thus being shifted by one line fromone frame to the next. Frame 32 is from line 32 to line 63. There are200 pixels per line, as noted above.

The incident light data for frame 1 to frame 32 is sequentiallytransferred to the memory 63 via the output processing circuit 62, andis stored in the memory 63, as shown in FIG. 13. More specifically, thememory 63 stores the incident light data for frame 1, frame 2, frame 3,. . . in this order. The data for line 32 contained in each frame isshifted upward by one line for each frame, that is, the 32nd line inframe 1, the 31st frame in frame 2, and so on. When the incident lightdata for frame 1 to frame 32 has been stored in the memory 63, thetemporal centroid Npeak is calculated for each pixel on line 32.

While the calculations are being performed for line 32, the incidentlight data for frame 33 is transferred to the memory 63 for storage. Asshown in FIG. 14, the incident light data for frame 33 is stored in thearea next to the area of the frame 32 in the memory 63. When the datafor frame 33 has been stored in the memory 63, the temporal centroidNpeak is calculated for each pixel on line 33 contained in frame 2 toframe 33.

While the calculations are being performed for line 33, the incidentlight data for frame 34 is transferred to the memory 63 for storage. Asshown in FIG. 15, the incident light data for frame 34 is stored in thearea where frame 1 was stored, and overwrites this previously storeddata. Since the data of frame 1 has already been processed, the data canbe safely erased by overwriting. When the data for frame 34 has beenstored in the memory 63, the temporal centroid Npeak is calculated foreach pixel on line 34. When the incident light data for frame 34 hasbeen processed, the incident light data for frame 35 is stored in anoverwriting fashion in the area where frame 2 was stored.

In this way, the temporal centroids Npeak are calculated for a total of200 lines, up to line 231, which is the last line. Of the incident lightdata stored in the memory 63, the data rendered unnecessary issequentially overwritten with new incident light data, as describedabove, thus serving to save the capacity of the memory 63.

The configuration of the centroid calculation circuit 73 will now bedescribed, along with the calculations of the temporal centroids Npeakperformed by the centroid calculation circuit 73. FIG. 16 is a blockdiagram showing the configuration of the centroid calculation circuit73, FIG. 17 is a diagram showing the concept of data transfer timing,and FIG. 18 is a diagram showing the concept of the temporal centroidNpeak.

As shown in FIG. 18, the temporal centroid Npeak is the centroid of the32 pieces of incident light data obtained by the 32 samplings. Samplingnumbers 1 to 32 are associated with the 32 items of incident light datafor each pixel. The i-th sample of incident light data is represented byxi, where i is an integer between 1 and 32. At this time, the index ifor a given pixel represents the number of frames processed after thepixel entered the effective light receiving region Ae.

The centroid ip of the 1st to 32nd incident light data x1 to x32 isobtained by dividing Si·xi, the summation of i·xi, by Sxi, the summationof xi. This is written as:${i\quad p} = \frac{\sum\limits_{i = l}^{32}{i*x\quad i}}{\sum\limits_{i = 1}^{32}{x\quad i}}$

The centroid calculation circuit 73 calculates the centroid ip (i.e.,temporal centroid Npeak) of each pixel based on the data read out of thememory 63. However, the data read from the memory 63 is not useddirectly, but the value obtained by subtracting steady-state ray data ksfrom each data sample is used (if the value is negative, 0 is used).That is, by subtracting the steady-state ray data ks, an offset is givento the incident light data output from the measuring sensor 53 c.

The steady-state ray data ks is the data calculated based on theincident light data of the pixel when the slit ray U is not incident onit. A predetermined fixed value may be used as the steady-state ray dataks, or alternatively, the data may be obtained in real time by usingdata output from the monochrome measuring sensor 53 c. When using afixed value, if the output of the monochrome measuring sensor 53 c is 8bits (256 gray scale levels), the value is set to “5”, “6”, or “10”, forexample. When obtaining the data in real time, the mean value of theincident light data for two pixels before and after the 32 samples ofincident light data for a given pixel is obtained, and the data with thesmaller mean value is taken as the steady-state ray data ks. The reasonis that the slit ray U is not incident on an area either before or afterthe effective light receiving region Ae and, therefore, the incidentlight data when the slit ray U is not incident can be reliably obtainedin real time. Further, of the incident light data for the two pixelsbefore and after a given pixel, the data with the larger mean value maybe taken as the steady-state ray data ks. Alternatively, the mean valueof the incident light data of the two pixels before the 32 samples ofincident light data, or the mean value of the incident light data of thetwo pixels after the 32 samples of incident light data, may be used.Incident light data for one pixel may also be used. Furthermore,depending on the shape of the object Q or the condition of the noisecontained in the incident light data, a value obtained by adding apredetermined value (for example, 5) to the above values may be used asthe steady-state ray data ks, thereby increasing the offset to ensurereliable elimination of unwanted noise components. In such cases, thoughthe size of one frame is 36 lines, 34 lines, or 33 lines, it is onlynecessary to use 32 samples of data for 32 lines for the calculation ofthe centroid ip.

Referring back to FIG. 16, the centroid calculation circuit 73 consistsof a steady-state ray data storing section 731, a subtracting section732, a first summing section 733, a second summing section 734, and adividing section 735. These sections are implemented using software, butit is also possible to construct all or part of them by hardwarecircuits.

The steady-state ray data storing section 731 stores the steady-stateray data ks. The subtracting section 732 subtracts the steady-state raydata ks from the input incident light data. The data output from thesubtracting section 732 is denoted as incident light data xi. The firstsumming section 733 sums i·xi for i=1 to 32, and outputs the value ofthe sum. The second summing section 734 sums xi for i=1 to 32, andoutputs the value of the sum. The dividing section 735 divides theoutput value of the first summing section 733 by the output value of thesecond summing section 734, and outputs the centroid ip. The centroid ipoutput from the dividing section 735 is stored in the display memory 74.The output value of the first summing section 733 and the output valueof the second summing section 734 are stored in output memories 64 a and64 b, respectively. The data stored in the output memories 64 a and 64 bare transmitted from the digital output terminal 33 to the host 3 viathe SCSI controller 66 or stored on the recording medium 4. At the host3, processing for three-dimensional position calculations is performedon the basis of this data, and also, the reliability of this data isjudged.

Referring to FIG. 17, the memory control circuit 63A sequentiallyspecifies addresses in the memory 63 for each pixel so that the centroidcalculation circuit 73 performs the above described processing for thepixel. For example, for line 32, the first line to be processed, anaddress is specified first for the data of the first pixel on line 32contained in frame 1, then for the data of the first pixel on line 32contained in frame 2, and so on, thus specifying addresses sequentiallyfor a total of 32 data samples from frame 1 to frame 32 for the firstpixel on line 32. By thus specifying the addresses, data is read fromthe memory 63 and transferred to the centroid calculation circuit 73.While the calculations are being performed for line 32, the incidentlight data for the next frame 33 is transferred into the memory 63. Forthe subsequent frames also, reading and writing of the memory 63 areperformed concurrently, thus achieving efficient circuit operation.

When the 32 samples of data have been input to the centroid calculationcircuit 73, the dividing section 735 outputs the centroid ip. Next,processing is performed on the data of the second pixel, then on thedata of the third pixel, and so on, until the 200th pixel is processed,thus completing the calculations of the centroids ip for line 32. Uponcompletion of the calculations of the centroids ip for line 32, thecalculations of the centroids ip are performed for line 33, then forline 34, then for line 35, and so on, until all 200 lines up to line 231are processed for the calculations of the centroids ip.

The centroids ip stored in the display memory 74 are displayed on thescreen of the LCD 21. Each centroid ip is related to the position of asurface port ion of the object Q being measured, with the value of thecentroid ip increasing as the distance from the position of the surfaceof the object Q to the three-dimensional camera 2 decreases, anddecreasing as the distance increases. Therefore, the distancedistribution can be presented by displaying a grayscale image using thecentroids ip as grayscale data.

The configuration shown in FIGS. 9 to 18 is also employed in the firstand second embodiments described in connection with FIGS. 3-4 and 5-6,respectively. If the displayed image is a color image in theseembodiments, the grayscale values calculated from the centroids are usedto control the intensity of the color displayed on each pixel.

In the present embodiment, a monochrome luminance image can be displayedon the display 3 b by using Sxi. That is, Sxi is the sum of the outputsfor the 32 frames. In one of the 32 frames, the slit ray reflected atthe subject is received (assuming the subject is within the measurablerange). Since most of the ambient light is cut off by the band passfilter 80 g, Sxi represents the sum of the slit ray componentsirradiated during the 32 frame periods. The sum of the slit raycomponents can likewise be obtained for all pixels. When Sxi for eachpixel is treated as the display luminance data of each pixel, the resultis a monochrome image (a monochrome image for the wavelength of the slitray).

If there are 13 bits available for the data range of Sxi, and 8-bit datais used to produce a monochrome luminance display, for example, then theremaining 8 bits in Sxi, after the three highest order bits and the twolowest order bits have been removed, might be used. The particular 8-bitdata to be used should be determined by considering the actual valuesfor Sxi.

The embodiments have been described for the case where a CCD is used asthe imaging device, but it will be appreciated that the range image andthe color image can also be captured if a CMOS imaging sensor or thelike is used as the imaging device.

What is claimed is:
 1. A three-dimensional measurement apparatuscomprising: an optical system which projects a reference beam having apredetermined wavelength toward a target object to be measured; a lightsensor which is sensitive to light in a range of wavelengths includingsaid predetermined wavelength and other wavelengths, and which receiveslight from said target object and produces output data relating thereto;calculating means for calculating a three-dimensional shape of saidtarget object based on first output data that is generated by said lightsensor on the basis of received light including said predeterminedwavelength; and generating means for generating two-dimensional imageinformation for said target object based on second output data from saidlight sensor that is based on received light of wavelengths excludingsaid predetermined wavelength.
 2. The three-dimensional measurementapparatus of claim 1, wherein said light sensor is a color measurementsensor.
 3. The three-dimensional measurement apparatus of claim 2,further including a light filter including an infrared cutoff portionand a band pass portion, and a filter switching mechanism forselectively placing said cutoff portion or said band pass portion in anoperative relationship with said light sensor in dependence upon whetherthe output data from said sensor is to be used by said display device orsaid calculating means, respectively.
 4. The three-dimensionalmeasurement apparatus of claim 1, wherein said light sensor is amonochromatic light sensor.
 5. The three-dimensional measurementapparatus of claim 4, further including a light filter having aplurality of color filter portions and a band pass portion, and a filterswitching mechanism for selectively placing said plurality of colorfilter portions or said band pass portion in an operative relationshipwith said light sensor in dependence upon whether the output data fromsaid sensor is to be used by said display device or said calculatingmeans, respectively.
 6. The three-dimensional measurement apparatus ofclaim 1, wherein said optical system includes a scanning device whichscans the reference beam across the target object, and said light sensorproduces multiple data samples of each pixel of an image as saidreference beam is being scanned, and further including a centroidcalculation circuit which calculates the centroid for each pixel on thebasis of said multiple data samples, wherein said display devicedisplays said image as a grayscale image based on said centroids.
 7. Thethree-dimensional measurement apparatus of claim 6, wherein each datasample from said light sensor comprises a frame containing apredetermined number of lines of an image, and said centroid calculationcircuit calculates the centroid for each pixel on a line once saidpredetermined number of frames have been sampled.
 8. Thethree-dimensional measurement apparatus of claim 7, wherein the centroidfor each pixel on a line is calculated while the data for the next framefollowing said predetermined number of frames is being sampled andstored in a memory.
 9. A method for determining the three-dimensionalshape of an object and displaying an image of the object, comprising thesteps of: projecting a reference beam having a predetermined wavelengthtoward an object to be measured; receiving light from the object bymeans of a light sensor which is sensitive to light in a range ofwavelengths including said predetermined wavelength and otherwavelengths; generating output data from said light sensor indicative ofthe amount of light received from the object; calculating informationrelating to the three-dimensional shape of the object on the basis offirst output data from said light sensor that is based on received lightincluding said predetermined wavelength; and generating two-dimensionalimage information for said target object based on second output datafrom said light sensor that is based on received light of wavelengthsexcluding said predetermined wavelength.
 10. The method of claim 9,further including the steps of selectively filtering light received bysaid light sensor with an infrared cutoff filter or a band-pass filterin dependence upon whether the data generated by said light sensor is tobe used to display said image or to calculate said three-dimensionalshape, respectively.
 11. The method of claim 9, further including thesteps of selectively filtering light received by said light sensor witha plurality of color filters or with a band-pass filter in dependenceupon whether the data generated by said light sensor is to be used todisplay said image or to calculate said three-dimensional shape,respectively.
 12. The method of claim 9, further including the steps ofscanning said reference beam across said object, generating multipledata samples of each pixel of an image as said reference beam is beingscanned, calculating a centroid for each pixel on the basis of saidmultiple data samples, and displaying said image as a grayscale imagebased on said centroids.
 13. The method of claim 12, wherein said lightsensor produces successive data samples each comprising a framecontaining a predetermined number of lines of an image, and the centroidfor a pixel on a line is calculated once said predetermined number offrames have been sampled.
 14. The method of claim 13, wherein thecentroid for each pixel on a line is calculated while the data for thenext frame following said predetermined number of frames is beingsampled and stored in a memory.
 15. A three-dimensional measurementapparatus comprising: a scanning light projection system; a single lightsensor which receives light reflected from a target object; a processorwhich determines a three-dimensional shape of said target object basedon output data from said light sensor in a first operation mode; adisplay device which displays an image of said target object on thebasis of output data from said sensor in a second operation mode; and atleast one optical filter which is selectively disposed in an opticalpath of said light sensor in dependence upon the operation mode.
 16. Thethree-dimensional measurement apparatus of claim 15, wherein said lightsensor is a color measurement sensor.
 17. The three-dimensionalmeasurement apparatus of claim 15, wherein said light sensor is amonochromatic light sensor.
 18. A three-dimensional informationmeasurement apparatus comprising: an optical system for projecting areference beam toward a target object to be measured; a light sensor forreceiving light reflected from said target object; and a controller forproducing first and second sets of information, said first set ofinformation being related to a three-dimensional shape based on anoutput of said sensor while said optical system projects the referencebeam toward the target, and said second set of information being relatedto a two-dimensional shape based on output of said sensor during aperiod of time that said optical system does not project the referencebeam.
 19. The three-dimensional measurement apparatus of claim 18,wherein said controller controls said optical system and said lightsensor to produce said first and second sets of information at differenttimes.
 20. The three-dimensional measurement apparatus of claim 18,wherein said controller positions a filter in front of the light sensorin a light path when at least either said first set of information isproduced or said second set of information is produced.
 21. Thethree-dimensional measurement apparatus of claim 20, wherein saidcontroller positions the filter in front of said light sensor when thefirst set of information is produced, and said filter is capable oftransmitting only light which has same wavelength as said referencebeam.